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C. M. Llodvall and Richard C. Cox 

* • 

The evaluation of an educational Innovation can serve two 
basic and related purposes. It can serve as an aid in the development 
of the innovation , and it can provide a basis for making judgments 
with respect to it. In the development and trial of an innovation 
it is Important that all components be monitored quite continuously 
so that rather immediate feed-back can be used to suggest any worthwhile 
modifications. The continuous monitoring of the program can also 
serve to investigate many specific contingencies that are basic to 
• the plan. That is, it permits the examination of relationships 
represented by such statements as, "If we do this, then the result 
should be this," or "If we modify this practice in this way, then 
the outcomes should be changed in this way." The developmental . 
functions that can be served by evaluation are probably its most 
important service. 

However, evaluative information is also essential for making 
decisio n s concerning whether or not a given innovation should *>e 
adopted or whether or not it should be continued after it has been 
given a trial. This is a matter of determining its value. Basic 
tp this step is some specification of the goals or the values that 
the innovation is intended to achieve and the identification of the 
related dependent variables that are to be assessed. 




Both of these purposes have been of concern in planning the 
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Individually Prescribed Instruction (IPI), a major project of the 
Learning Research and Development Center at the University of 
Pittsburgh. The remainder of this paper and the three papers that 
follow will attempt to illustrate how this concern has been 
operationalized . 

Overview of the IPI Procedure 

The IPI procedure is a program designed to achieve a certain 
type of individualized Instruction in grades K through 6 in the subject 
areas of reading , arithmetic , and science. Basically it has involved 
the development of extensive and detailed sequences of objectives in 
each subject area, the preparation of self-study materials that the 
student can use to master each objective, and a procedure for testing 
and for individual lesson planning which permit each student to progress 
through the curriculum sequence at his own pace. The procedure provides 
for a rather continuous monitoring of the progress of each pupil through 
the use of placement tests, unit tests, and curriculum embedded tests 
and the frequent and periodic development of individualized lesson 
plans or ''prescriptions*' for each pupil. A few of the basic aspects of 
the IPI procedure, and aspects which will be given further attention 
in our presentations here today, include the following: 

1* A major portion of pupil class time is spent in individual 
* work (study, test taking, consultation with teacher, etc.). 

2. Each pupil is working from an Individualized prescriptions 
based on his level of achievement and certain other factors. 
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3. Pupils progress at individual rates in such a way that 

at any given time almost every pupil will be at a different 
point in the curriculum sequence from that of his classmates « 

The Evaluation of the IPI Project 

To ensure that the evaluation plan for the Individually 
Prescribed Instruction procedure will be as complete as possible, it 
has been found useful to define it as follows. The IPI procedure is 
(1) a planned program (2) operating in specific ways (3) in some given 
school context (4) in order to achieve certain goals in pupil behavior 
and (5) teacher behavior and which may also (6) have certain unplanned 
Influences . So that our evaluation activities will be of as much 
use as possible, both in developing IPI and in examining its worth, 
we have given some thought to each of these six elements. 

(1) The Program Plan . Any Innovation involves a plan which 
is spelled out in writing and/or in the minds of persons. This plan 
is an independent variable in the sense that one must study the effect 
of it on the operation of the program and on the goals that the innovation 
is designed to achieve. For this purpose it is essential that the plan 
be described quite completely so that it is known just what this 
independent variable involves. 

The plan for Individually Prescribed Instruction is spelled 
out in a number of different documents, many of which have been 
assembled in our draft of a manual for training persons to use the 
procedure. Its key features are summarized in a brief paper which is 
considered to be somewhat definitive and which has the title, "The 
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Essential Elements of Individually Prescribed Instruction." These 
documents help to define IPI and provide the basis for subsequent steps 
in its assessment. 

Specifying the detailed plan of an IPI program that is put 
into operation in any school that is planning to adopt this procedure 
es an innovation should be a basic step in the evaluation program for 
that adoption. As an aid in assessing the extent to which any school 
preparing to adopt IPI actually has done the planning necessary for 
its operation, the staff has developed a questionnaire entitled, "IPI 
Evaluation Guide" which is to be used as one step in monitoring 
dissemination efforts. 

(2) The Operating Program. A second step in the evaluation 
of an educational Innovation must be a description of how the program 
actually looks in operation. What do pupils' do? What do teachers do? 
What materials and techniques are used? In assessing the IPI program 
this 6tep Involves securing answers to such questions as: Do pupils 
proceed at Individual paces? Do they work quite Independently for 
most of the time? Is the teacher providing individual help rather 
than lecturing?— and so on. 

The assessment of the operating program requires extensive 
observation of pupils and teachers. As a first step in this type of 
assessment a "Pupil Observation Record" has been developed and employed 
on a trial basis. Dr. Yeager will present a description of its use 
and of some results obtained. He will also enlarge upon the uses 




of such data. 
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Among other things, descriptions of operational aspects of 
IPI have been useful in the careful study of the results of certain 
procedures. For example, to study the prescription-writing process 
it has been Important to look at the relationship between the procedure 
followed by the teacher and the extent to which the prescriptions 
produced are varied to fit the needs of the student. Dr. Bolvln 
will present some data bearing on this specific problem. 

(3) The School Context . Any school program takas place within 
a context which has some impact on the way in which the program is 
operated. Although this context probably cannot be considered as 
either an independent or dependent variable, it will Influence the 
innovation, and any evaluation plan should make provision for a 

• description of Important elements in the context • The IPI evaluation 
staff has prepared a rather lengthy description of the school context 
within which the original development of IPI has taken place. This 
type of study of school context may be particularly important in the 
dissemination of the innovation in that certain qualities of the 
school, of its student body, or of the community may actually be 
important determiners of the success of the Innovation. 

(A) Pupil Behavior — (Variability in Pupil Achievement as a 
Dependent Variable) . The major dependent variable in essentially 
all instructional activity is some type of pupil achievement. Changes 
In what pupils and teachers do in the classroom must be important 
Intermediate goals in most instructional Innovations, but such changes 

* are only means to an end, namely, pupil achievement. This is true of 
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the IPI project. Individually Prescribed Instruction Is a planned 
procedure involving specific types of activities on the part of pupils 
and teachers but its ultimate accomplishments are to be measured largely 

i * 

in terms of measures of pupil achievement. For this reason major 
variables that must be assessed are those involving achievement measures. 

A major feature of IPI is its detailed specification of the 
sequences of instructional objectives through which each pupil is to 
proceed at his individual pace. Pupil progress through these sequences 
is monitored and guided through the use of special tests specifically 
designed to measure the stated objectives. Hence, the most valid 
criteria of pupil achievement are the placement tests, the unit tests, 
and the curriculum embedded tests that are an integral part of the 
planned program. 

Of course, the major objective of the program is the 
individualization of instruction. Defining the objective in this 
vay has important implications for -evaluation. It means that the 
basic step in evaluating IPI in terms of pupil achievement must be 
to examine pupil variability. Do students actually vary in their 
levels of achievement and in their rates of progress? Is this 
variability greater than that which is achieved under other methods 

of instruction? Recent data on this variable will be presented 

* 

in my (Dr. Cox's) paper which will be given later. 

(5) Teacher Behavior . Teachers functioning in an Individually 
Prescribed Instruction program must use skills that are somewhat 
different from those used by teachers in more conventional classrooms. 
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Such skills can be considered both as dependent and Independent variables. 
The extent to which teachers develop and use these abilities may be 
an important determiner of pupil progress and achievement. For this 
reason a study of the relationship between teacher mastery of these 
skills and student behavior and progress is an important step in the 
development of the IPI project, both at Oakleaf and in any schools to 
which the program is disseminated. However, the teachers' acquisition 
of these skills can also be looked upon as an Important outcome of 
the IPX plan and of the procedures that are a basic part of the IPI 
.operation (e.g., studying test results, writing prescriptions, taking 
part in staff conferences). In this sense these skills represent 
dependent variables that should be studied in evaluating the effectiveness 

t 

of the plan and the operation. 

Thus far, our assessment of teacher behavior has been done 
quite informally, but specific and relatively objective procedures 
are now being developed. 

(6) Unplanned Influence on Other Variables . With any type 
of Innovation it is Important to be concerned not only with its 
influence on those variables that it is designed to affect but also 
with its Influence on other variables. This is important both in 
terms of its possible positive and negative effects. A rather radical 
departure from typical school practice such as is represented by the 
IPI procedure is likely to Influence both pupils and teachers in a 
variety of ways. 
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Some variables of this type that have been investigated in 
a relatively informal manner include pupil attendance, teacher 
satisfaction, and parental acceptance. Others will be examined in 
the future. 

Xhis, in rather broad outline, is out evaluation plan and 
its rationale. The other staff members of the IPI project will enlarge 
upon specific activities that we have undertaken within the context 
of this overall rationale. 



